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Abstract 



■ This paper illustrates how a Prolog program, using chronological backtracking to find a 

solution in some search space, can be enhanced to perform intelligent backtracking. The 
enhancement crucially relies on the impurity of Prolog that allows a program to store 
information when a dead end is reached. To illustrate the technique, a simple search 
^ ' program is enhanced. 
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1 Introduction 

The performance of backtracking algorithms for solving finite-domain constraint 
satisfaction problems can be improved substantially by so called look-back and 
look- ahead methods l|Dechter and Frost 2002JI . Look-back techniques extract infor- 
mation by analyzing failing search paths that are terminated by dead ends and use 
that information to prune the search tree. Look-ahead techniques use constraint 
propagation algorithms in an attempt to avoid such dead ends altogether. Con- 
straint propagation can rather easily be isolated from the search itself and can be lo- 
calized in a constraint store. Following the seminal work of ; Van Hentenryck~1 989), 
look-ahead techniques are available to the logic programmer in a large number of 
systems. 

This is not the case for look-back methods. Intelligent backtracking has been ex- 
plored as a way of improving the backtracking behavior of logic programs ( |Bruynooghe and Pereira 1984| ) . 
For some time, a lot of effort went into adding intelligent backtracking to Prolog im- 
plementations (see references in ( |Bruynooghe 1991] )). However, the inherent space 
and time costs, which must be paid even when no backtracking occurs, impeded its 
introduction in real implementations. 

For a long time, look-ahead methods dominated in solving constraint satisfaction 
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problems. However, already in ( |Rosiers an d Bruynoog he 1987| ) we have shown em- 
pirical evidence that look-back methods can be useful, even that it can be interesting 
to combine both. Starting in the nineties there is a renewed interest in look-back 
methods, e.g., ( |Ginsberg 1993| ), and in combining look-back with look-ahead e.g., 
IjDechter and Frost 2002jl . 

Look-back turned out to be the most successful of the approaches tried in a 
research project aiming at detecting unsolvable queries (queries that do not termi- 
nate, such as the query «— odd(X), even(X) for a program defining odd and even 
numbers). The approach was to construct a model of the program over a finite 
domain in which the query was false. The central part of this model construction 
was to search for a pre-interpretation leading to the desired model, i.e., with D 
the domain, to find an appropriate function D n — > D for every n-ary functor in 
the program. A meta-interpreter was built which performed a backtracking search 
over the solution space. A control strategy was devised which resulted in the early 
detection of instances of program clauses which showed that the choices made so 
far could not result in the desired model. This meta-interpreter outperformed ded- 
icated model generators on several problems flBruynooghe et al. 1998| ). However it 
remained very sensitive to the initial ordering in which the various components of 
the different functions were assigned. The point was that not all choices made so 
far necessarily contributed to the evaluation of a clause instance. We experimented 
with constraint techniques and also investigated the use of intelligent backtracking. 
With a small programming effort, we could enhance the meta-interpreter to sup- 
port a form of intelligent backtracking. As reported in pSruynooghe et al. 1 999), 
this was the most successful approach. As Prolog is a popular tool for prototyp- 
ing search problems and as look-back methods, though useful, are not available in 
off-the-shelf Prolog systems, we decided to describe for a wider audience how to 
enhance a Prolog search program with a form of intelligent backtracking. The tech- 
nique crucially depends on the impure feature of Prolog (assert/retract) that allows 
storing information when a dead end is reached. The stored information is used to 
decide whether a choice point should be skipped when chronological backtracking 
returns to it. Hence we propose the technique as a black pearl. 

In the application mentioned above, the meta-interpreter is performing a sub- 
stantial amount of computation after making a choice whereas the amount of com- 
putation added to support intelligent backtracking is comparatively small. This is 
not always the case. When the amount of computation in between choices is small 
and solutions are rather easy to find, the overhead of supporting intelligent back- 
tracking may be larger than the savings due to the pruning of the search space. 
This is the case in toy problems such as the n-queens. In the example we develop 
here, there is a small speed-up. 

We recall some basics of intelligent backtracking in Section In Section we 
introduce the example program and in Section 0] we enhance it with intelligent 
backtracking. We conclude with a discussion in Sectional 
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2 Intelligent Backtracking 

Intelligent backtracking as described in flBruynooghe 198T1 ) is a very general schema. 
It keeps track of the reason for eliminating a variable in a domain. Upon reaching 
a dead end, it identifies a culprit for the failure and jumps back to the choice point 
where the culprit was assigned a value. Information about the variables assigned 
in between the culprit and the dead end can be retained if still valid, as in the dy- 
namic backtracking of ( |Ginsberg 1993| ) which can be considered as an instance of 
the schema. More straightforward in a Prolog implementation is to give up that in- 
formation, this gives the backjumping algorithm (Algorithm 3.3) in ( |Ginsberg 1993| ) 
(intelligent backtracking with static order in (Ros iers and Bruynooghe"l 987)). We 
follow rather closely ( |Ginsberg"l 993) for introducing it. 

A constraint satisfaction problem (CSP) can be identified by a triple (I,D, C) 
with / a set of variables, D a mapping from variables to domains and C a set of 
constraints. Each variable i G / is mapped by D into a domain Di of possible values. 
Each constraint c G C defines a relation R c over a set I c C I of variables and is 
satisfied for the tuples in that relation. A solution to a CSP consists of a value Vi 
(an assignment) for each variable i in I such that: (1) for all variables i: Vi G Di 
and, (2) for all constraints c: with I c = {ji, . . . ,jk}, it holds that (v^ , . . . , Vj k ) G R c - 

A partial solution to a CSP (I, D, C) is a subset J C I and an assignment to each 
variable in J. A partial solution P is ordered by the order in which the algorithm 
that computes it assigns values to the variables and is denoted by a sequence of 
ordered pairs (i, Uj). A pair (i, v») indicates that variable i is assigned value Vi] 
Ip = {i\(h v i) G P} denotes the set of variables assigned values by P. 

Given a partial solution P, an eliminating explanation (cause-list in ( [Bruynooghe 1981} ) 
for a variable i is a pair S) where Vi G Di and S C Ip. It expresses that the 
assignments to the variables of S by the partial solution P cannot be extended into 
a solution where variable i is assigned value «j. Contrary to (Gi nsberg 1993| ), we use 
an elimination mechanism that tests one value at a time. Hence we assume a func- 
tion consistent(P, i, Vi) that returns true when PU{(i, Vi)} satisfies all constraints 
over Ip U {i}) and a function elim{P, i, vi) that returns an eliminating explanation 
(vi, S) when -^consistent(P ', i,Vi). 

Below, we formulate the backjumping algorithm; next we clarify its reasoning. 
Ei is the set of eliminating explanations for variable i. 

Algorithm 1 

Given as inputs a CSP (J, D, C). 

1. Set P := 0. 

2. If Ip = I return P. Otherwise select a variable i G I \ Ip, set Si := Di and 
#i := 0. 

3. If Si is empty then go to step 4; otherwise, remove an element Vi from it. 

If const stent(P, i, Vi) then extend P with (i, «j) and go to step 2; otherwise 
add elim(P, i, Vi) to Ei and go to step 3. 

4. (Si is empty and Ei has an eliminating explanation for each value in Di.) Let 
C be the set of all variables appearing in the explanations of Ei . 
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5. If C = 0, return failure. Otherwise, let (I, vi) be the last pair in P such that 
I 6 C . Remove from P this pair and any pair following it. Add (vi, C \{l}) 
to Ei, set i :— I and go to step 3. 

In step 3, when the extension of the partial solution is inconsistent then elim(P ', i, Vi 
returns a pair (vi,{jx, . . . ,j m }) such that the partial solution (ji, Vj ± ), . . . , (j m , Vj m ), (i, 
violates the constraints. The inconsistency of this assignment can be expressed by 
the clause: <— ji = Vj 1 ,...,j m = Vj m ,i — Vi (The head is false, the body is a 
conjunction). 

In step 4, when Si is empty, we have an eliminating explanation for each value 
Vi k in the domain Di . Hence we have a set of clauses of the form 

<- Jfc,l = %,i , • • ■ ,jk,m k = %, mjt = (1) 
The condition that the variable i must be assigned a value from domain Di with 
n elements can be expressed by the clause (the head is a disjunction, the body is 
true): 

i = v il ,...,i= v ln <- (2) 

Now, one can perform hyperresolution (Rob inson 1965J) between clause (J3Jl and 
the clauses of the form (QJ (for k from 1 to n). This gives: 

< ^71,1 1 ' ' ■ ( Jl,TOi — v ji,m 1 7 • ■ ■ i Jn,l — w in,i' ' ' ' )3n,m„ = Vj n ,m H (3) 

This expresses a conflict between the current values of the variables in the set 
Oi,li • • • jii.mn • • • )in,l) • • ■ j jn.m,,} = C- Hence, with Z the last assigned variable 
in C, C \ {1} is an eliminating explanation for The conflict C is computed in 
step 4. When empty, the problem has no solution as detected in step 5. Otherwise, 
step 5 backtracks and adds the eliminating explanation C \ {I}) to the set of 
eliminating explanations of variable I. 

One can observe that the algorithm does not use the individual eliminating ex- 
planations in the set Ei — (vi k , Sk), but only the set C which is the union of the sets 
Sk- As we have no interest in introducing more refined forms of intelligent back- 
tracking, we develop Algorithm [2] where Ei holds the union of the sets Sk in the 
eliminating explanations of variable i. To obtain an algorithm that closely corre- 
sponds to the Prolog encoding we present in Section we reorganise the code and 
introduce some more changes. The function elim(P, i, Vi) that returns an eliminat- 
ing explanation [vi, S) for the current value of variable i is replaced by a function 
conflict(P,i,Vi) that returns the set {i} U S (the variables that participate in a 
conflict as represented by Equation [JJ. This conflict is stored in a variable C (step 
3 of Algorithm [5J . It is nonempty and i is the last assigned variable, hence the 
value of i remains unchanged in step 4 and, in step 5, the eliminating explanation 
C \ {i} is added to Ei. This reorganisation of the code has as result that a local 
conflict (the chosen value for the last assigned variable i is inconsistent with the 
partial solution) and a deep conflict (all values for variable i have been eliminated) 
are handled in a uniform manner: upon failure, the algorithm computes a conflict 
and stores it in variable C (for the local conflict in step 3, for the deep conflict in 
step 5), backtracks to the variable computed in step 4 (the "culprit") and resumes 
in step 5 with updating Ei and trying a next assignment to variable i. 
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Algorithm 2 

Given as input a CSP (I, D, C). 

1. Set P := 0. 

2. If Ip = I return P. Otherwise select a variable i G I\ Ip. Select a value Vi 
from Di. Set # := D t \ {v t } and E, := 0. 

3. If consistent(P , i, Wj) then extend P with (i, «j) and go to step 2; otherwise 
set C := conflict(P, i,Vi). 

4. If C = then return failure; otherwise let (7, «;) be the last pair in P such 
that / e C. Set i := Z. 

5. Add C \ {i} to Ei. If 5, = then C := ^ and go to step 4; otherwise select 
and remove a value Vi from S'j and go to step 3. 

3 A search problem 

The code below is, apart from the specific constraints, fairly representative for a fi- 
nite domain constraint satisfaction problem. The problem is parameterized with two 
cardinalities: VarCard, the number of variables (the first argument of problem/3) 
and ValueCard, the number of values in the domains of the variables (the second 
argument of problem/3). The third argument of problem/3 gives the solution in 
the form of a list of elements assign{i, v t ). The main predicate uses init_domain/2 
to create a domain [1,2, . . . ,ValueCard\ and init_pairs/3 to initialize Pairlist 
as a list of pairs i-Di with Di the domain of variable i. The first argument of 
extend_solution/3 is a list of pairs i-Di with i an unassigned variable and Di 
what remains of its domain; the second argument is the (consistent) partial so- 
lution (initialized as the empty list) and the third argument is the solution. The 
predicate is recursive; each iteration extends the partial solution with an assignment 
to the first variable on the list of variables to be assigned. The nondetcrministic 
predicate my_assign/2 selects the value. If desirable, one could introduce a selection 
function which dynamically selects the variable to be assigned next. 

Consistency of the new assignment with the partial solution is tested by the pred- 
icates consistentl/2 and consistent2/2. They create a number of binary con- 
straints. The binary constraints themselves are tested with the predicates constraintl/2 
and constraint2/2. What they express is not so important. The purpose is to cre- 
ate a problem that is sufficiently difficult so that enhancing the program with intelli- 
gent backtracking pays off. For the interested reader, the predicate consistent2/2 
creates a very simple constraint that verifies (using constraintl/2) that the value 
of the newly assigned variable is different from the value of the previously assigned 
variable. The predicate consistentl/2 creates a set of more involved constraints. 
The odd numbered and even numbered variables each encode the constraints of the 
n-queens problem. As a result, the solution of e.g., problem(16,8,S) contains a 
solution for the 8-queens problem in the odd numbered variables and a different 
(due to the constraints created by consistent2/2) solution in the even numbered 
variables. Substantial search is required to find a first solution. For example, the 
first solution for problem (16, 8, S) is found after 32936 assignments (using a simi- 
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lar set-up of constraints, a solution is found for the 8-queen problem after only 876 
assignments). 

Note that the constraint checking between the new assigned variable and the 
other assigned variables is done in an order that is in accordance with the order 
of assigning variables. Hence consistent 1/2 is not tail recursive. The order is not 
important for the algorithm without intelligent backtracking. However, it is cru- 
cial to obtain optimal intelligent backtracking: as with chronological backtracking, 
constraint checking will stop at the first conflict detected and an eliminating expla- 
nation will be derived from it. As an eliminating explanation with an older assigned 
variable gives more pruning than one with a more recently assigned variable, the 
creation of constraints requires one to pay attention to the order. It is done already 
here to minimize the differences between this version and the enhanced version. 

problem (VarCard, ValueCard, Solution) : - 
init_domain(ValueCard, Domain) , 
init_pairs (VarCard , Domain , Pairs) , 
extend_solution(Pairs, [] .Solution) . 

init_domain (ValueCard, Domain) :- 

( ValueCard=0 -> Domain=[] 
; ValueCard>0, ValueCardl is ValueCard- 1, 
Domain= [ValueCard I Domainl] , 
init_domain (ValueCardl , Domainl) 

). 

init_pairs (VarCard, Domain, Vars) : - 
( VarCard=0 -> Vars = [] 
; VarCard>0, VarCardl is VarCard- 1, 
Vars= [VarCard-Domain I Varsl] , 
init_pairs (VarCardl .Domain, Varsl) 

). 

extend_solution( [] , Solution, Solution) . 

extend_solution( [Var-Domain I Pairs] ,PartialSolution, Solution) :- 
my_assign (Domain, Value) , 

consistentl (PartialSolution, assign(Var , Value) ) , 
consistent2 (PartialSolution, assign(Var , Value) ) , 
extend_solution(Pairs, 

[assign(Var, Value) I PartialSolution] , 

Solution) . 

my_assign( [Value I _] .Value) . 

my_assign( [_ I Domain] .Value) :- my_assign(Domain, Value) . 
consistentl ([],_). 



Programming pearl 



7 



consistentl ( [_] , _) . 

consistentl ( [_ , Assignmentl I PartialSolution] .AssignmentO) :- 
consistentl (PartialSolution, AssignmentO) , 
constraintl (AssignmentO , Assignmentl) , 
constraint2 (AssignmentO , Assignmentl) . 

consistent2( [],_). 

consistent2( [Assignmentl I _] .AssignmentO) :- 

constraintl (AssignmentO .Assignmentl) . 

constraintl (assign(_ ,ValueO) ,assign(_,Valuel)) :- ValueO \== Valuel. 

constraint2(assign(VarO,ValueO) , assign (Varl , Valuel) ) :- 
Dl is abs(ValueO-Valuel) , 
D2 is abs(VarO-Varl)//2, 
Dl \== D2. 

4 Adding intelligent backtracking 

Adding intelligent backtracking requires us to maintain eliminating explanations. 
In Algorithm [21 a single eliminating explanation is associated with each variable. 
The eliminating explanation of a variable i is initialised as empty in step 2, when 
assigning a first value to the variable. It is updated in step 5, when the last assigned 
value turns out to be the "culprit" of an inconsistency. This happens just before 
assigning the next value to variable i. This indicates that the right place to store 
eliminating explanations is as an extra argument in the predicate my_assign/2. 
In step 4, the algorithm has to identify the "last" variable I of a conflict (the 
"culprit"), just before updating the eliminating explanation. We will also use the 
my _as sign/2 predicate to check whether the variable it assigns corresponds to the 
culprit of the failure. Hence also the identitity of the variable should be an argument. 
These considerations lead to the replacement of the my_assign/2 predicate by the 
following my_assign/4 predicate. 

my_assign( [Value | _] ,_Var,_Explanat ion, Value ) . 
my_assign( [_ I Domain] ,Var,ExplanationO, Value) :- 

get_conflict (Conflict) , 

remove (Var , Conflict , Explanat ionl) , 

set_union(ExplanationO ,Explanationl .Explanation) , 

my_assign (Domain , Var , Explanation , Value ) . 
my_assign([] ,_Var, Explanat ion, _Value) :- 

save_conf lict (Explanation) , fail. 

It is called from extend_solution/4 as myassign (Domain, Var , [] .Value) (what 
remains of the domain is the first argument, the second argument is the variable 
being assigned, the third argument is the initially empty eliminating explanation 
and the fourth argument returns the assigned value) . The initial call together with 
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the base case perform the otherwise branch of step 2. The second clause, entered 
upon backtracking when the domain is nonempty, checks whether the variable being 
assigned is the culprit. To do so, it needs the conflict. As this information is com- 
puted just before failure occurs, it cannot survive backtracking when using the pure 
features of Prolog. One has to rely on the impure features for asserting/updating 
clauses. Either assert/1 and retract/1 or more efficient variants of specific Prolog 
systems 1 . The call to get_conflict (Conflict) picks up the saved conflict 2 ; next, 
the call remove(Var, Conflict, Explanationl) checks whether Var is part of it. If 
not, my _as sign/4 fails and backtracking returns to the previous assignment. If Var 
is the culprit, then the code performs step 5 of the algorithm: remove/3 returns 
the eliminating explanation in its third argument, set_union/3 adds it to the cur- 
rent eliminating explanation and the recursive call checks whether the domain is 
empty. If not, the base case of my _as sign/4 assigns a new value. If the domain is 
empty, then the last clause is selected. The eliminating explanation becomes the 
conflict and is saved with the call to save_conf lict (Explanation) that relies on 
the impure features 3 and the clause fails. 

Further modifications are in the predicates constraint 1/2 and constraint2/2 
that perform the constraint checking. If a constraint fails, the variables involved in 
it make up the conflict and have to be saved so that after re-entering my as sign/4 
the conflict can be picked up and used to compute an eliminating explanation (step 
3). As the last assigned variable participates in all constraints, it is part of the 
conflict. For example, the code for constraint 1/2 becomes: 

constraint 1 (assign (VarO,ValueO) ,assign(Varl,Valuel)) :- 
( ValueO \== Value 1 -> true 
; save_conf lict( [VarO,Varl] ) , fail 
). 

The modification to constraint2/2 is similar. Recall that the order in which 
constraints are checked determines the amount of pruning that is achieved. Finally, 
if one is interested in more than one solution then also a conflict has to be stored 
when finding a solution. It consists of all variables making up the solution. Us- 
ing a predicate allvars/2 that extracts the variables from a solution, the desired 
behavior is obtained as follows: 

problem (VarCard, ValueCard, Solution) : - 
init_domain(ValueCard, Domain) , 
init_pairs (VarCard , Domain , Pairs) , 
extend_solution(Pairs , [] , Solution) , 
initbacktracking(Solution) . 

initbacktracking (Solution) :- 

allvars (Solution, Conflict) , 

1 In our experiments, we made use of SICStus Prolog and employed bb_put/2 and bb_get/2. 

2 We implemented it as get.conf lict (Conflict) :- bb_get (conflict , Conflict) . 

3 We implemented it as save.conf lict (Conflict) :- bb_put(conf lict, Conflict). 



Programming pearl 



9 



save_conf lict (Conflict) . 

The enhanced program generates the same solutions as the original, and in the 
same order. For problem ( 16 , 8 , S) the number of assignments goes down from 32936 
to 4015 and the execution time from 140ms to 70ms; for problem (20 , 10 , S) , the re- 
duction is respectively from 75950 to 15813 and from 370ms to 310ms. The achieved 
pruning more than compensates for the (substantial) overhead of recording and 
updating conflicts 4 and of the calls to remove/3 and set_union/3. Note that the 
speed-up decreases with larger instances of this problem. This is likely due to the 
increasing overhead of the latter two predicates. Keeping the conflict set sorted 
(easy here because the variable numbers corresponds with the order of assignment) 
such that the culprit is always the first element could reduce that overhead. 

5 Discussion 

In this black pearl, we have illustrated by a simple example how a chronological 
backtracking algorithm can be enhanced to perform intelligent backtracking. As 
argued in the introduction, look-back techniques are useful in solving various search 
problems. Hence exploring their application can be very worthwhile when building 
a prototype solution for a problem. The technique presented here illustrates how 
this can be realized with a small effort when implementing a prototype in Prolog. 
Interestingly, the crucial feature is the impurity of Prolog that allows the search 
to transfer information from one point in the search tree (a dead end) to another. 
It illustrates that Prolog is a multi-faceted language. On the one hand it allows 
for pure logic programming, on the other hand it is a very flexible tool for rapid 
prototyping. Note that the savings due to the reduction of the search space could 
be undone by the overhead of computing and maintaining the extra information, 
especially, when the amount of computation between two choice points is small. 

The combination of look-back and look-ahead techniques can be useful, and algo- 
rithms integrating both can be found, e.g., IfDechter and Frost 2002J1 . The question 
arises whether our solution can be extended to incorporate look-ahead. This re- 
quires some work, however, much of the design can be preserved. The initialization 
(init_domains/3) should not only associate variables with their initial finite do- 
main, but also with their eliminating explanations (initially empty). Then the code 
for the main iteration could be as follows: 

extend_solution( [] , Solution, Solution) . 

ext end_solut ion (Var s , Part ialSolut ion, Solution) : - 

selectbestvar (Vars ,var(Var, Values , Explanation) ,Rest) , 
myassign ( Values , Var .Explanation, Value) , 
consistent (PartialSolution, assign(Var , Value) ) , 
propagate( [assign(Var, Value) I PartialSolution] , 
NewPart ialSolut ion) 

4 Using bb_get and bb_put to count the number of assignments increases execution time of the 
initial algorithm for problem (16, 8, S) from 140ms to 400ms. 
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ext end_solut ion ( Var s .NewPartialSolut ion, Solution) . 

The predicate selectbestvar/3 is used to dynamically select the next variable to 
assign. It returns the identity of the variable (Var), the available values (Values) 
and the explanation (Explanation) for the eliminated values. When a partial so- 
lution is successfully extended, the predicate propagate/2 has to take care of the 
constraint propagation: eliminating values from domains and updating the cor- 
responding explanations after which the next iteration can start. Computing the 
eliminating explanation for each eliminated value requires great care and depends 
on the kind of look-ahead technique used. It is pretty straightforward for forward 
checking but requires careful analysis in case of e.g., arc consistency as no pruning 
will occur on backjumping when the elimination is attributed to all already assigned 
variables. 
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